Universum-Inspired Supervised Contrastive Learning

Mixup is an efficient data augmentation method which generates additional samples through respective convex combinations of original points and labels. Although being theoretically dependent on properties, reported to perform well as a regularizer calibrator contributing reliable robustness generalization neural network training. In this paper, inspired by Universum Learning uses out-of-class assist the target tasks, we investigate from largely under-explored perspective - potential generate in-domain that belong none classes, is, universum. We find in framework supervised contrastive learning, universum-style produces surprisingly high-quality hard negatives, greatly relieving need for large batch size learning. With these findings, propose Universum-inspired Contrastive learning (UniCon), incorporates strategy universum g-negatives pushes them apart anchor classes. Our approach not only improves with labels, but also innovates novel measure data. linear classifier learned representations, Resnet-50, our achieves 81.68% top-1 accuracy CIFAR-100, surpassing state art significant margin 5% much smaller size.